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(54) METHOD AND DEVICE FOR RETRIEVAL OF DYNAMIC IMAGE 

(57)Abstract: r-?— 
PURPOSE: To provide a new storage and management *~ ~ 
method for dynamic image information which retrieves 
an intended scene from a long-time dynamic image 
stored in a recording medium in a short time. 
CONSTITUTION: With respect to dynamic image 
information which manages and retrieves the dynamic 
image, an entire dynamic image R is hierarchically 
divided into shorter-time frame strings A, A1, A11... in 
accordance with its constitution and semantic contents to 
generate tree structure data 10, 13, 15, 16, and 17 
indicating their hierarchical relations, attribute data At of 
frame strings, and still pictures rf of representative 
frames, and they are related to one another to generate 
a data file. Attribute data or the hierarchical tree is used 
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as the retrieval key to extract a frame string from this data file of dynamic image information, 
and contents are confirmed by the still picture (rf) of its representative frame, thus retrieving 
the dynamic image. Since random access to an arbitrary scene is possible and related 
scienes are easily retrieved, an intended dynamic image is quickly retrieved from the long- 
time dynamic image. 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] The management method of the dynamic image characterized by what the hierarchy tree data 
with which it is an approach for retrieving the dynamic-image information accumulated in dynamic- 
image management equipment, and carrying out the extract playback of the dynamic image in a record 
medium, and the above-mentioned dynamic-image information describes the logical structure which 
composed each frame of the above-mentioned dynamic image in a frame train based on a physical 
change of the above-mentioned dynamic image or change of a semantic content, the static-image data of 
the frame representing each above-mentioned frame train, and the attribute data of each above- 
mentioned frame train are included for. 

[Claim 2] It is the approach of searching the dynamic image in a record medium with dynamic-image 
management equipment and an interactive I/O means equipped with the memory which accumulated 
dynamic-image information, and a retrieval means. The above-mentioned dynamic-image information 
The frame string data which composed each frame of the above-mentioned dynamic image in two or 
more frame trains based on a physical change of the above-mentioned dynamic image or change of a 
semantic content, It sets in the above-mentioned retrieval section including the static-image data of the 
frame representing each above-mentioned frame train, and the attribute data of each above-mentioned 
frame train. The search method of the dynamic image characterized by what the frame train containing 
the above-mentioned attribute data is searched, and the static-image data of the representation frame of 
the extracted above-mentioned frame train are displayed for on the above-mentioned interactive I/O 
means. 

[Claim 3] The above-mentioned dynamic-image information is the search method of the dynamic image 
according to claim 2 characterized by what a frame train is searched for using the above-mentioned 
hierarchy tree data and the above-mentioned attribute data in the above-mentioned retrieval section 
including the hierarchy tree data which describe the logical structure which composed each frame of the 
above-mentioned dynamic image in the frame train based on a physical change of the above-mentioned 
dynamic image or change of a semantic content. 

[Claim 4] It is the approach of generating the dynamic-image information which manages a dynamic 
image with dynamic-image management equipment equipped with an interactive I/O means, the 
dynamic-image information generation section, and memory. By the above-mentioned dynamic-image 
information generation section In order to compose each frame of this dynamic image in two or more 
frame trains and to describe the logical structure of each above-mentioned frame train by a physical 
change of the above-mentioned dynamic image or change of a semantic content The management 
method of the dynamic image characterized by what a hierarchy tree is created, the frame representing 
each above-mentioned frame train is defined, the static-image data is created, and is recorded on the 
above-mentioned memory with the attribute data of each frame train. 

[Claim 5] The management method of the dynamic image according to claim 4 characterized by the 
above-mentioned dynamic-image information generation section by what the hierarchy tree which 
described the logical structure of each above-mentioned frame train is created for while compose each 
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frame of this dynamic image in two or more frame trains automatically, organization of the above- 
mentioned frame train is displayed on the above-mentioned interactive I/O means, it waits for a user 
input and organization of the above-mentioned frame train is changed by a physical change of the 
above-mentioned dynamic image or change of a semantic content. 

[Claim 6] It is the approach of managing a dynamic image using dynamic-image management 
equipment equipped with a dynamic-image storing means, a dynamic-image playback means, a display 
means, the dynamic-image information generation section, the retrieval section, and memory. By the 
above-mentioned dynamic-image information generation section By a physical change of the above- 
mentioned dynamic image or change of a semantic content, each frame of this dynamic image is 
composed in two or more frame trains. The frame representing each above-mentioned frame train is 
defined, the static-image data is created, and it records on the above-mentioned memory with the 
attribute data of each frame train. By the above-mentioned retrieval section The management method of 
the dynamic image which extracts the above-mentioned frame train from the above-mentioned memory, 
displays the static image of the representation frame of the above-mentioned frame train on the above- 
mentioned display means, and is characterized by what the extracted above-mentioned frame train is 
reproduced for in the above-mentioned dynamic-image playback means by making the above-mentioned 
attribute data into a retrieval item. 

[Claim 7] The management method of the dynamic image according to claim 6 characterized by what a 
hierarchy tree is created, it records on the above-mentioned memory with the attribute data of a frame 
train, and the above-mentioned frame train is extracted for from the above-mentioned memory in the 
above-mentioned retrieval section by the above-mentioned dynamic-image information generation 
section by making the above-mentioned attribute data and a hierarchy tree into a retrieval item in order 
to describe the logical structure of each above-mentioned frame train. 

[Claim 8] It is retrieval equipment of the dynamic image equipped with a means to store a dynamic 
image, a display means, and the retrieval section. The above-mentioned retrieval section The frame 
string data which composed each frame of a dynamic image in two or more frame trains by a physical 
change of the above-mentioned dynamic image or change of a semantic content, The memory which 
recorded the static-image data of the frame representing each above-mentioned frame train, and the 
attribute data of each frame train, Retrieval equipment of the dynamic image characterized by what the 
above-mentioned attribute data extracted the frame train in the above-mentioned memory, and it has a 
retrieval means to display the static image of the above-mentioned representation frame on the above- 
mentioned display means for. 

[Claim 9] The above-mentioned retrieval means is retrieval equipment of the dynamic image according 
to claim 8 characterized by what the data and attribute data of the above-mentioned hierarchy tree 
extract the above-mentioned frame train for including the data of hierarchy Thurs. by which the memory 
of the above-mentioned retrieval section described the logical structure of each above-mentioned frame 
train. 

[Claim 10] It is dynamic-image management equipment equipped with the means, the interactive I/O 
means, and the dynamic-image information generation section which store a dynamic image. The above- 
mentioned dynamic-image information generation section A frame train generation means composes in 
a frame train for every scene, and generate a frame train by which each frame of the above-mentioned 
dynamic image was changed, Dynamic-image management equipment which comes to have a 
representation frame creation means to create the attribute data of each above-mentioned frame train, to 
define the frame representing each frame train, and to give the static-image data. 
[Claim 1 1] The above-mentioned dynamic-image information generation section is dynamic-image 
management equipment according to claim 10 which comes to have a frame train re-division and a 
hierarchy tree creation means to create a hierarchy tree while re-merging based on the inclusion relation 
of a frame in order to describe the logical structure of the above-mentioned frame train to be the scene 
change detector which detects change of the scene of the above-mentioned dynamic image. 
[Claim 12] The above-mentioned hierarchy tree creation means is dynamic-image management 
equipment according to claim 1 1 characterized by what each frame of the above-mentioned dynamic 
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image is automatically composed as a frame train based on inclusion relation, this frame train is 
displayed on the above-mentioned interactive I/O means, it is based on a user input, a frame train is 
divided and merged, and a hierarchy tree is created for about this frame train. 
[Claim 13] It is equipment which manages a dynamic image using dynamic-image management 
equipment equipped with a dynamic-image storing means, a dynamic-image playback means, a display 
means, the dynamic-image information generation section, and the retrieval section. The above- 
mentioned dynamic-image information generation section A frame train generation means to compose 
each frame of the above-mentioned dynamic image in a frame train for every scene which changed, The 
frame representing each above-mentioned frame train is defined, the static-image data is created, and it 
has a representation frame creation means to give the attribute data of each frame train. The above- 
mentioned retrieval section Dynamic-image management equipment which comes to have the memory 
which recorded the above-mentioned dynamic-image information generated in the above-mentioned 
dynamic-image information generation section, and a retrieval means to extract the above-mentioned 
frame train from the above-mentioned memory, to display on the above-mentioned display means, and 
to reproduce this frame train with the above-mentioned dynamic-image playback means. 
[Claim 14] In order that the above-mentioned dynamic-image information generation section may 
describe the logical structure of the above-mentioned frame train to be the scene change detector which 
detects change of the scene of the above-mentioned dynamic image It has a hierarchy tree creation 
means to create a hierarchy tree while it is based on the inclusion relation of a frame and a frame train is 
divided and merged. The above-mentioned retrieval section Dynamic-image management equipment 
according to claim 13 which comes to have a retrieval means to extract the above-mentioned frame train 
from the above-mentioned memory by making the above-mentioned attribute data and a hierarchy tree 
into a retrieval item, to display on the above-mentioned display means, and to reproduce this frame train 
with the above-mentioned dynamic-image playback means. 



[Translation done.] 
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JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2 **♦* s hows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention relates to the management technique of the dynamic image for 
searching at random the dynamic image accumulated in the record medium, and reproducing and 
displaying. 
[0002] 

[Description of the Prior Art] The image equipment for the record medium which recorded the dynamic 
image, for example, a laser disc, VTR, 8mm video, etc. spreads, and the accumulated dose of an image 
has been increasing remarkably also at the museums and the ordinary homes including a special field of 
study of a broadcasting station, an advertising agency, etc. Reuse of it not only reproducing the 
accumulated image, but it making a new image by edit is also performed frequently. The are recording 
management method of the dynamic-image information which can discover the target image scene 
efficiently from a record medium in playback, edit, etc. is needed as are recording of an image becomes 
huge. This is the same also in the movie field for an image film. 

[0003] It refers to the management equipment of the conventional dynamic image by accumulating the 
frame number used as an index in storages, such as a personal computer, and a user specifying the 
accumulated frame number. For example, a personal computer accumulates a frame number in a storage. 
A user displays the static image of a frame with the frame number in which a frame number and frame 
time were directly specified from the alphabetic character numeric-value input unit, or the personal 
computer was accumulated on a display, and when a user chooses from an input unit, he specifies a 
reproductive initiation frame. A personal computer is reproduced on TV monitor from the specified 
initiation frame, controlling coma delivery which an image reproduction machine has, and functions, 
such as a rapid traverse and rewinding. Thus, by the conventional method, the approach of carrying out 
which carries out direct access of the image used as the candidate for retrieval with a frame number etc. 
is taken. 

[0004] It is based on the management method of such a dynamic image, and they are Takafumi 
Miyatake work, an "interactive natural animation edit technique", and FRIEND 21. The technique of 
carrying out the outline of the 3rd meeting for announcing the results and the scene which divides a 
dynamic image into a scene then, displays the image of the head frame of each scene on a display on 
July 18 or 19, 1991, and serves as a candidate for edit is shown. However, when [ which was aimed at 
the dynamic image of long duration ] it averages at the time and change of a scene arises by about 1 time 
of frequency at 2 seconds, the number of scenes increases and it becomes difficult to cany out the 
outline of all those scenes efficiently. Moreover, it cannot search, grasping the configuration of the 
whole dynamic image as the semantic configuration of a scene becomes complicated, since the divided 
scene is the direct unit of retrieval. 

[0005] On the other hand, in order to search a dynamic image with the information retrieval equipment 
shown in JP,6 1-29939 ,A, a dynamic image is classified hierarchical and the technique which displays as 
a menu the static image which symbolizes a classification hierarchical, and a user chooses is shown. 
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Retrieval becomes however, less efficient, since a dynamic image is displayed only after it follows the 
hierarchy of a classification from the most significant to the least significant in detail as the hierarchy of 
a menu becomes deep. Moreover, the data control technique for the hierarchy of a menu to be fixed and 
change this is not shown. 
[0006] 

[Problem(s) to be Solved by the Invention] The purpose of this invention is to offer the are recording 
management method of the new dynamic-image information for searching the scene meant from the 
dynamic image of the long duration accumulated in the record medium for a short time. 
[0007] 

[Means for Solving the Problem] According to this invention, by a physical change of a dynamic image 
or change of a semantic content, each frame of a dynamic image is composed in two or more frame 
trains, and the dynamic-image information for searching each frame train is created. The attribute data 
used as the retrieval item of the static-image data of the frame representing the hierarchy tree which 
described the logical structure of each above-mentioned frame train, and each above-mentioned frame 
train, and each frame train is contained in this dynamic-image information. By using this dynamic-image 
information for retrieval of a dynamic image, random access is attained to the scene of arbitration, and 
since retrieval of the scene moreover related also becomes easy, the dynamic image meant from the 
dynamic image of long duration can be searched quickly. 
[0008] 

[Example] The conceptual diagram of dynamic-image management which used the dynamic-image 
information on this invention for drawing 1 is shown. The dynamic image 12 which consists of many 
(for example, per second 30 pieces) frames fl-fh is divided into two or more short-time frame trains 14 
by a physical change of frames, such as a cut and a switch of camera angle, and change of a semantic 
content rather than an original dynamic image, as shown in drawing (a). The break of each frame train 
14 is relative, and arbitrary. For example, a certain frame train is further divided into the assembly of a 
short-time frame train, and Japanese lacquer and the frame train which plurality follows conversely can 
be merged, and it can be considered more semantically that it is the frame train of a high order. In order 
to describe the logical structure of the frame train based on the inclusion relation of such a frame f, the 
hierarchy tree 10 as shown in (b) of drawing is created. The dynamic image whole [ 12 ] corresponds to 
the root node (R) 15 of the hierarchy tree 10, and the frame train 14 divided and merged corresponds to 
an intermediate node 16 and a leaf node 17. Moreover, the arc 13 expressed with the adjacency of the 
upper and lower sides of a node expresses the parentage of a frame train. It sets to arbitration about the 
frame train corresponding to each node of the hierarchy tree 10, one or more frames (rf) 18, i.e., 
representation frame, representing the frame train, and the static-image data is created. And dynamic- 
image information is completed by storing the reference pointer to a representation frame image (rf) in 
each node of a hierarchy tree with the attribute data (At) 19 used as the retrieval item of a frame train. 
[0009] As shown in (a) of drawing 1 , this hierarchy tree detects automatically change of the frames fl- 
fh in the dynamic image whole [ 12 ], divides it into the frame train 14 of a smallest unit like Al 1 and 
A 12, and generates one hierarchy's tree structure. Next, from the frame train of these smallest units, a 
user creates many hierarchies* tree structure in bottom-up by the contiguity frame train to which the 
contents relate being suitably merged like Al 1 and A12 to Al. Or as shown in drawing 2 , the tree 
structure is created in top-down by repeating dividing the accumulated dynamic image whole [ 12 ] into 
the frame train 14 of arbitration like A, B, and C by decision of OPETA, and dividing each frame train 
into the frame train (for example, AA1, A2, A3) of short-time arbitration more further. Many 
hierarchies' tree structure 10 is created editing a frame train based on the contents of the scene in any 
[ this ] case (re-division and re-merge). 

[0010] Retrieval of a dynamic image is performed based on migration of the node which met collating 
and the arc 13 of the attribute data 19 of a node (15, 16, 17) in the hierarchy tree 10 using dynamic- 
image information. That is, as retrieval conditions, the attribute (At) of a frame train is specified or the 
conditions which follow a hierarchy tree are specified, such as looking for the frame train corresponding 
to nodes, such as parents, a child, twin, etc. of the assignment node in the hierarchy tree 10. The 
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representation frame 18 and attribute data 19 of a frame train are displayed as a retrieval result, and 
about the frame train 14 which the user chose from the inside, a dynamic image is accessed and it 
reproduces. 

[001 1] Drawing 3 shows the dynamic-image management equipment of the bottom up system which 
becomes one example of this invention. The dynamic image 20 is stored in a record medium like a laser 
disc (Following LD and brief sketch) as an analog image. 21 is an animation playback means and is 
equipped with the laser disc player (following LD player) 22, the TV monitor 23, and the analog / digital 
converter 24. 25 is a scene change detector. Although 26 is the retrieval information generation section 
which performs generation processing of various kinds of information required for retrieval management 
of a dynamic image and may be constituted from hardware of dedication, combination with the program 
expressing a personal computer and generation procedure (it mentions later by drawing 4 ) can also 
constitute it. The retrieval information generation section 26 is equipped with the representation frame 
creation means 27, the hierarchy tree creation means 28, memory 29, and the dialogue input-control 
means 30. Memory 29 has the representation frame image file 3 1 , the change frame number file 32, and 
the hierarchy tree data file 33. 

[0012] 34 is a dialogue input means and has the input means 35 and displays 36 for an alphabetic 
character and numeric values, such as a keyboard, and a mouse, a touch panel. Although 37 is the 
retrieval section and you may constitute from hardware of dedication, combination with the program 
expressing a host computer and retrieval procedure (it mentions later by drawing 10 ) can also 
constitute. The retrieval section 37 is equipped with the are recording means 38, the retrieval means 39, 
and the storage means 40 of data, and the storage means 40 is equipped with the representation frame 
image file 41 and the hierarchy tree data file 42. 

[0013] In the configuration of drawing 3 , the scene change detector 25 changes the analog image from 
the LD player 22 into a digital image by the analog / digital transducer 24, scene change is detected 
automatically, the list of frame numbers of Frame f from which the scene changed is outputted, and this 
is recorded on the change frame number file 32 of the retrieval information generation section 26. The 
representation frame creation means 27 defines the representation frame (rf) 1 8 about each frame train 
which becomes settled from the list of this frame number, captures the rest frame image of a 
representation frame from the LD player 22 through an analog / digital converter 24, creates an image 
file, and records it on the representation frame image file 3 1 . Moreover, the hierarchy tree creation 
means 28 displays the rest frame image obtained from the frame number list and the LD player 22 of the 
change frame number file 32 on a display 36, and creates the hierarchy tree 10 which stored the attribute 
information At about a frame train based on the input of the user from the input means 35. This 
representation frame image 1 8 is accumulated in the representation frame image file 3 1 , and the 
hierarchy tree 10 is accumulated in the hierarchy tree data file 33. 

[0014] Next, the example of the procedure which stores data in memory 29 with the equipment of 
drawing 3 about the frame train of a dynamic image is explained according to drawing 4 . First, in the 
1st step 41, the scene change detector 25 detects scene change on the basis of a physical change of a 
frame etc., and outputs the frame number of the frame f which changed to the change frame number file 
32. since it can come out and said Takafiimi Miyatake work carried out, an "interactive natural 
animation edit technique", and the shown well-known technique can be used about the scene change 
detector 25, for example, the detailed explanation is omitted. In step 42, the hierarchy tree creation 
means 28 calculates initiation / termination frame number of the frame train 14 from the list of frame 
numbers which a scene change detector outputs. Thereby, as an initial state of the hierarchy tree of a 
frame train, the dynamic image whole [ 12 ] is made into a parent node, and the tree structure 10 of one 
hierarchy who makes a leaf node the frame train 14 divided by the scene change detector 25 is created. 
In step 43, the hierarchy tree creation means 28 displays the hierarchy tree of an initial state on a display 
36. By directing edit (division and merge) actuation of a hierarchy tree from the alphabetic character 
numeric-value input means 35, a user changes the structure of an early hierarchy tree and builds many 
hierarchies 1 tree. 

[0015] Next, in step 44, the representation frame creation means 27 determines the frame number of the 
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representation frame 18 about each frame train. Let representation frames be frames of the location set 
up beforehand, such as a head frame, a frame which carried out fixed frame number progress of for 
example, a frame train. About the determined frame rf, an analog / digital conversion is carried out from 
LD22, and a static-image file is created. In step 45, a user corrects the representation frame 
automatically set up at this step if needed. At this time, the frame newly used as a representation frame 
is directed, when a user makes the TV monitor's 23 image stand it stil! with that frame. In step 46, a user 
inputs the value of the attribute item At which describes the contents of the frame trains, such as a title 
and a keyword, etc. about the frame train corresponding to each node of a hierarchy tree. These attribute 
items are used as a key at the time of retrieval. The attribute data and the representation frame image 
about the frame train created at the above step are accumulated in the files 31 and 32 of memory 29 in 
step 47. 

[0016] In addition, in the above-mentioned explanation, an analog / digital converter 24, and the TV 
monitor 23 become unnecessary by being able to omit the processing which changes an analog image 
into a digital image, when a dynamic image is stored as a digital image, and displaying a dynamic image 
on a display. Moreover, this configuration is a personal computer 26 and a host computer 37. It is also 
possible to replace and carry out by two or more workstations connected by LAN, respectively. 
[0017] Next, the detail of creation of the dynamic-image information (a representation frame image and 
hierarchy tree) accumulated in the files 31 and 32 of memory 29 or the files 41 and 42 of memory 40 
and the example of retrieval is explained based on a drawing. First, the structure of the file for storing 
dynamic-image information, such as a hierarchy tree of the frame train accumulated in a host's memory 
29 and 40 and a representation frame image, is explained. 

[0018] Drawing 5 shows a frame number for an axis of abscissa 51, and shows the example of the 
hierarchy tree 10 for an axis of ordinate 52 as a hierarchy's depth. The dynamic image whole [ 12 ] 
corresponds to the root node (R) 1 5. Moreover, the frame train A is divided with Frames fa and fb, and 
has the child frame trains Al and A2 and A3. The child frame train Al has the grandchild frame trains 
Al 1 and A 12 further. Moreover, the frame number of a representation frame is shown about each frame 
train by the location which projected the mark 53 of the representation frame rf on the shaft 51 of a 
frame number. For example, each of representation frames of R, A, and A2 is rf(s)l. 
[0019] Drawing 6 is the example of the file structure for storing such dynamic-image information, (a) is 
the structure of the hierarchy tree data file (33 42) which stores the attribute data of a frame train, and 
describes the identifier 54 of a frame train, the initiation frame number 55, and the termination frame 
number 56 to each frame train 14 used as the node of a hierarchy tree. Furthermore, the value 57 of the 
attribute item (Atl 1, Atl2 ...) which describes the contents of the frame trains, such as a title and a 
keyword, etc. more than a piece at least, and the reference pointer 59 to the static-image file (31 41) of 
the frame number 58 of a representation frame and a representation frame are stored in one record, the 
identifier 54 of a frame train — for example, the initiation frame number 55 and the termination frame 
row number 56 — constructing — etc. ~ from — the value which identifies a frame train uniquely is 
assigned. The hierarchical relationship of a frame train stores in a hierarchy tree data file (33 42) the 
record which constructed and **(ed) the identifier 61 of a parent frame train, and the identifier 62 of a 
child frame train, as shown in (b). 

[0020] In step 43 of drawing 4 , modification of the hierarchy tree 10 is based on the actuation which 
divides a frame train and creates a child frame train, and the actuation which two continuous frame 
trains are merged and creates a parent frame train. As actuation for that, the frame train A as shown in 

(A) of drawing 7 is divided. The actuation 70 which creates the child frame trains Al and A2, the 
actuation 71 which creates the child frame train B which makes parents the frame train A as shown in 

(B) , and the frame train B as shown in (C) are divided. The twin frame train Bl as shown in the 
actuation 72 which creates the twin frame trains B and Bl, and (D), and B-2 are merged, and the 
actuation 73 which creates the parent frame train B is specified, or — or these twist operations 74-77 are 
specified. When a new frame train is created by division and a merge of these frame trains, the record 
about the frame train is added to the file of drawing 6 . Moreover, when a new parentage is created by 
division of a frame train, in a data file (33 42), **** of the identifier of the frame train of the parents and 



http://www4.ipdl.inpit.go.jp/cgi-bin/tran_web_cgi_ejje 



11/26/2007 



JP,05-282379,A [DETAILED DESCRIPTION] 



Page 5 of 6 



child is added to the identifier 61 of a parent frame train, and the identifier 62 of a child frame train. 
[0021] In modification actuation of the hierarchy tree of step 43 of drawing 4 , the representation frame 
of the child frame train newly [ when a frame train is divided ] created is automatically set up based on 
the representation frame of a parent frame train. In the example shown in drawing 8 , when the frame 
train A which has a frame fl in a representation frame is divided in a frame f2 (<fl), the situation before 
and after setting up the pointer to the representation frame number and image file of the child frame 
trains Al and A2 which are created by division is shown. The representation frame of A is fl and has 
the reference pointer 80 to an image file II. Since the frame £2 to divide has a front frame number from 
fl , the representation frame of Al turns into the head frame f3 by setup of default, and the representation 
frame of A2 is set to the fl [ same ] as A. Moreover, the pointer 8 1 to the image file of A2 points out the 
same address as the pointer 82 of A. Although it becomes a null pointer 83 about Al, when the image 
file of a frame D is created, it transposes to the address of the file, on the contrary, the case where the 
frame trains Al and A2 are merged to A — Al and A2 — the direction which the user chose among each 
representation frame serves as the representation frame rf of A. 

[0022] In step 45 of drawing 4 , modification of the representation frame of a certain frame train affects 
the both sides of the parent frame train and a child frame train along with the hierarchy of a frame train. 
The example of drawing 9 shows a situation before and after changing the representation frame of a 
frame train with a child frame train. When the representation frame fl of the frame train A is changed 
into f2 (<fl), since a frame f2 is included, the representation frame of Al is also interlocked with A, and 
the child frame train Al of A is changed into f2. With this, the pointer 85 to the image file of Al comes 
to point out the same address as the pointer 84 of A. Modification of the representation frame of this 
parent frame train affects a child frame train similarly to the frame train used as a leaf node. 
Modification of the representation frame of the frame train A affects the parent frame train at it, when 
the representation frame of the frame train which serves as parents of A is contained by coincidence at 
A. Modification of the representation frame of this child frame train affects a parent frame train 
similarly to the frame train of a root node. 

[0023] Next, the processing at the time of the retrieval using the retrieval section 37 is explained below 
by drawing 10 . At the time of retrieval, the appointed screen of retrieval conditions as shown in drawing 
1 1 is first displayed on the display means 36. A user inputs the retrieval conditions about animation 
information from the input means 35 (step 102). That is, the value made into retrieval conditions at the 
criteria specification column 91, respectively is inputted to each attribute item column 90 about a frame 
train to search, and the retrieval activation carbon button 92 is pushed. The retrieval means 39 performs 
retrieval processing to the file structure ( drawing 6 (a)) which stored the attribute data of the frame train 
currently held at the files 41 and 42 of memory 40 based on the retrieval condition etc. (step 104), and 
indicates the frame train of relevance by the output as a retrieval result at a display 36. That is, the image 
list of the representation frames rf of the frame train which corresponds to retrieval conditions as shown 
in the example of a screen of drawing 12 is displayed (step 106). 

[0024] When the representation frame rf to search to an image list is not contained, it can return to an 
initial screen ( drawing 11 ), i.e., the appointed screen of retrieval conditions, by pushing the carbon 
button 93 of attribute modification (step 108). the attribute data At concerning the frame train when the 
representation frame rf to search is displayed and the field is directed and chosen (step 1 10), as the 
example of a screen of drawing 13 shows a representation frame list — the representation firame image rf 
— table ** last (step 1 12) . In the screen of drawing 13 , when a frame train is found, and a user pushes 
the playback carbon button 96, the control signal with which the retrieval means 39 reproduces the 
directed frame train is sent to the LD player 22. The LD player 22 sends the analog image stored in 
LD20 to the TV monitor 23, and indicates this by playback (step 1 16). Moreover, it can return to the 
screen ( drawing 12 ) of a representation frame list with the list carbon button 97 to see other 
representation frame images rf (step 120). 

[0025] Moreover, a push on the hierarchy retrieval carbon button 98 gives a menu indication of the 
selection carbon button 99 of retrieval actuation which met the hierarchy tree, as the example of a screen 
of drawing 14 shows (step 122). Each selection carbon button 99 is for specifying actuation of searching 
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frame trains, such as parents, a child, and a twin frame, on the basis of the frame train displayed on 
drawing 13 , respectively, the child frame trains [ as opposed to / search an identifier 62 from the 
identifier 61 by the side of the parent frame train of a file 42 about the frame train displayed on drawing 
13 when a child frame train was searched from a parent frame train and the "child" of the selection 
carbon button 99 was pushed, for example, / the parent frame train A ] Al and A2 is extracted (step 
124). In searching a grandchild frame train, it carries out two-times activation of the search to an 
identifier 62 from an identifier 61 as opposed to a child frame train from a parent frame train. In 
searching "twin" frame train, it searchs a child frame train with the same parent frame train as an 
identifier 61 from an identifier 62. In pushing "parents" and searching a parent frame train from a child 
frame train, it searchs an identifier 61 from the identifier 62 by the side of a child frame train. The result 
of these retrieval is displayed as a screen ( drawing 12 ) of the list of representation frames. 
[0026] The top-down method stated by drawing 2 is also realizable with the equipment of drawing 3 . 
However, in this case, the scene change detector and the change frame number file are unnecessary, and 
using the representation frame creation means 27, the hierarchy tree creation means 28, and the dialogue 
I/O means 34, a user creates a hierarchy tree and a representation frame by dialogic operation, and does 
record maintenance of them at memory 30. The search method is the same as said example. 
[0027] Since this invention can be carried out also with the combination of hardware like throats, such 
as a host computer, a personal computer, and a visual equipment, and both digital image and analog 
image are made as for it to an object, the applicability is wide. 
[0028] 

[Effect of the Invention] According to this invention, as dynamic-image information for managing and 
searching a dynamic image, the whole dynamic image is **(ed) to the configuration and semantic 
content, it divides hierarchical as a short-time frame train more, and the tree structure data in which the 
hierarchical relationship is shown, the attribute data of a frame train, and the static image of a 
representation frame are generated. By using such dynamic-image information as a search key, the 
frame train of arbitration can be made random or it is easily accessible to a related frame train. It is 
effective in the ability to search a short scene in a short time rather than it had semantics from the 
dynamic image of long duration by this. 
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1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
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